Moment-Matching and Best Entropy Estimation

Spectral moment matching in the maximum entropy spectral analysis method

The formula for the spectral moments is expanded in a series consisting of autocorrelation terms. By using the autocorrelation extrapolation inherent in the maximum entropy method (MEM) for spectral analysis, it is found that good estimates of the moments can be found from a very low order autoregressive model. The motivation for this investigation came from results obtained during spectral ana...

Chebyshev polynomials, moment matching, and optimal estimation of the unseen

We consider the problem of estimating the support size of a discrete distribution whose minimum non-zero mass is at least 1 k using independent samples. We show that the minimax sample complexity to achieve an additive error of k with probability at least 0.5 is within universal constant factors of k log k log 2 1 , which improves the state-of-the-art result k 2 log k due to Valiant and Valiant...

Accurate Multi-segment Probability Density Estimation Through Moment Matching

The impact of process variations continues to grow as transistor feature size shrinks. Such variations in transistor parameters lead to variations and unpredictability in circuit output and may ultimately cause them to violate specifications leading to circuit failure. In fact, timely failures in critical circuits may lead to catastrophic failures in the entire chip. As such, statistical modeli...

Moment - Entropy Inequalities

Generative Moment Matching Networks

We consider the problem of learning deep generative models from data. We formulate a method that generates an independent sample via a single feedforward pass through a multilayer preceptron, as in the recently proposed generative adversarial networks (Goodfellow et al., 2014). Training a generative adversarial network, however, requires careful optimization of a difficult minimax program. Inst...